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ABSTRACT 


This  report  contains  a  survey  of  the  operation  and  properties  of  various 
types  of  joint  transform  correlators,  as  a  precursor  to  their  consideration  as 
the  central  component  in  analogue  optical  precision  image  registration  sys¬ 
tems.  The  exact  meaning  of  the  term  “precision  image  registration”  is  artic¬ 
ulated.  At  the  very  least,  an  analogue  optical  system  should  have  the  ability 
to  variably  magnify  and  rotate  images  if  it  is  to  have  prospects  of  executing 
precision  image  registration.  Motivated  by  this,  an  appraisal  of  various  optical 
techniques  of  image  rotation  is  undertaken.  These  investigations  lead  to  an 
objective  assessment  of  the  feasibility  of  conducting  precision  image  registra¬ 
tion  by  suitably  enhanced  joint  transform  correlators.  Qualitative  comparison 
between  analogue  optical  and  digital  electronic  techniques  of  precision  image 
registration,  favours  the  latter  as  the  methodology  for  a  practical  system. 
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Use  of  joint  transform  optical  correlators  for  precision 

image  registration 

EXECUTIVE  SUMMARY 

Image  registraiion  is  the  process  of  determining  the  transformation  between  two  images 
of  the  same  scene,  and  then  implementing  this  transformation  to  yield  two  images  that 
are  views  of  the  same  scene  from  the  same  perspective,  but  possibly  at  different  times. 
This  report  emphasises  the  first  step  of  this  process,  that  is,  the  determination  of  the 
transformation.  Image  registration  is  essential  to  a  wide  range  of  real  time  surveillance 
applications,  such  as  target  tracking.  Analogue  optics  is  being  actively  investigated  for 
use  in  automatic  target  detection  and  tracking,  its  appeal  being  primarily  fast  operation 
from  compact  hardware.  This  report  considers  whether  analogue  optical  techniques  can 
achieve  the  precision  required  for  high  quality  image  registration,  while  still  maintaining 
their  speed  advantages. 

Practitioners  often  adopt  a  minimalist  approach  to  image  registration,  whereby  the 
two  images  are  assumed  to  be  translated  with  respect  to  one  another,  but  otherwise  are 
identical.  The  standard  mathematical  operation  of  correlation  in  principle  will  determine 
exactly  the  translation  vector  components.  Correlations  can  be  efficiently  computed  by 
Fourier  techniques,  which  makes  this  type  of  image  registration  particularly  tractable. 

However,  there  are  circumstances  in  which  the  transformation  between  the  two  im¬ 
ages  is  not  accurately  approximated  by  a  pure  translation.  Rather,  the  transformation 
is  parametrised  by  extra  parameters  beyond  translation;  such  as  rotation,  magnification, 
shear,  and  nonlinear  geometric  terms  as  well.  For  sufficiently  wide  field  of  view  imaging 
systems,  the  transformation  even  scales  the  image  intensity  in  a  nonuniform  way,  thus 
requiring  even  more  elaborate  parametrisation.  In  such  circumstances,  image  registration 
becomes  a  much  more  formidable  problem,  and  one  must  take  a  much  more  sophisticated 
approach  towards  it.  For  the  purposes  of  this  report,  the  label  “precision  image  registra¬ 
tion”  will  be  applied  to  the  problem  of  registering  images  with  sufficiently  high  fidelity, 
that  the  transformation  components  additional  to  translation  become  important. 

Since  there  are  now  more  transformation  parameters  to  be  determined  than  the  two 
translation  components,  the  computational  effort,  and  hence  elapsed  time,  is  increased 
appreciably.  This  report  examines  the  radical  proposal  of  attempting  to  use  analogue 
optical  techniques  to  accomplish  precision  image  registration.  The  allure  of  analogue 
optics  resides  in  the  immense  speed  afforded  by  its  massively  parallel  operation. 

Analogue  optical  correlation  is  directly  implemented  by  a  variety  of  joint  transform 
correlators.  But  on  its  own,  a  joint  transform  correlator  can  only  hope  to  implement  the 
minimalist  approach  to  image  registration.  This  report  discusses  the  embellishments  that 
are  needed  on  a  conventional  joint  transform  correlator,  to  make  it  a  serious  proposition 
as  a  precision  image  registration  system.  Actually,  it  transpires  that  even  suitably  en¬ 
hanced  joint  transform  correlators  make  awkward  precision  image  registration  systems. 
On  balance,  digital  computation  seems  preferable. 
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1  Introduction 

Image  registration  is  a  substantial  and  active  field  of  image  processing  research  (refer 
to  Brown  [1]  for  a  review).  At  its  most  simplistic,  image  registration  involves  the  determi¬ 
nation  of  the  pure  translation  that  best  transforms  one  image  into  another.  This  is  often 
quite  adequate,  especially  if  one  of  the  images  is  partioned  into  small  segments,  each  of 
which  is  independently  matched  to  the  other  image.  Image  transformation  by  translation 
only  is  the  most  frequently  used  type  of  registration;  with  subpixel  precision  claimed  for 
this  operation  under  favourable  circumstances  (see,  for  example,  Tian  and  Huhns  [2]). 

However,  in  general  the  transformation  between  two  images  of  the  same  scene  is  not  a 
pure  translation,  but  rather  a  translation  plus  other  geometric  warping  components.  The 
exact  nature  of  the  image  warping  due  to  imaging  system  rotation  has  been  elucidated  by 
Caprari  [3].  Caprari’s  research  demonstrates  quite  emphatically  that  there  are  credible 
circumstances  in  which  pure  translation  is  a  poor  approximation  to  the  transformation 
between  images.  Under  these  circumstances,  high  fidelity  image  registration  necessarily 
incorporates  additional  geometrical  warping.  For  the  purposes  of  this  article,  the  term 
“precision  image  registration”  will  be  interpreted  as  meaning  image  registration  of  such  high 
fidelity,  that  geometric  image  warping  is  an  essential  element  of  the  operation,  together 
with  translation. 

Actually,  Caprari’s  research  also  demonstrates  that  for  wide  field  of  view  (>100°) 
imaging  systems,  there  is  also  a  significant  radiometric  image  warping  that  accompanies  the 
geometric  image  warping.  To  simplify  matters,  this  article  will  consider  only  registration  of 
image  pairs  for  which  the  radiometric  warping  is  insignificant;  that  is,  narrow  or  moderate 
field  of  view  images. 

Precision  image  registration  is  such  a  sophisticated  operation,  that,  to  the  author’s 
knowledge,  it  is  exclusively  implemented  as  non-real  time  computations  on  powerful  digi¬ 
tal  computers.  The  primary  benefit  of  conducting  precision  image  registration  by  digital 
computer,  is  the  ability  to  implement  any  conceivable  computational  algorithm,  given  suf¬ 
ficient  time.  For  example,  in  principle  there  is  no  reason  for  a  digital  computer  registration 
algorithm  to  exclude  the  complications  of  radiometric  warping.  The  primary  inadequacy 
of  traditional  digital  computers  is  the  elapsed  time  for  the  registration  operation  to  be 
completed,  by  essentially  sequential  computations.  It  is  true  that  a  reasonably  compact 
parallel  computer  simultaneously  executes  several,  possibly  even  many,  parallel  streams 
of  sequential  operations.  But  even  this  mode  of  computation  hardly  qualifies  as  being 
massively  parallel,  since  the  preponderance  of  complexity  of  the  overall  operation  resides 
in  the  sequential  procedure. 

The  purpose  of  this  article  is  to  explore  the  possibility  of  executing  precision  image  reg¬ 
istration  by  analogue  optical  techniques,  in  lieu  of  digital  techniques.  Correlation,  which 
is  the  basis  of  all  registration  operations,  can  be  implemented  directly  by  analogue  optical 
techniques,  by  a  variant  of  the  ubiquitous  JTC^.  The  benefit  of  JTC  image  registration 
is  its  potential  for  very  fast  response  time,  due  to  the  entirely  parallel  nature  of  optical 
correlation.  Conventional  JTCs  only  explore  pure  image  translations  in  their  correlation 

^JTC  =  Joint  Transform  Correlator 
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operation.  Herein  lies  the  major  problem  of  JTC  registration:  its  lack  of  flexibility  in 
exploring  more  general  image  transformations.  Substantial  enhancements  to  the  conven¬ 
tional  JTC  architecture  somewhat  broaden  the  scope  of  image  transformations  that  are 
explored  by  JTCs,  but  they  never  attain  the  supreme  flexibility  of  digital  computers.  For 
example,  it  would  be  horrendously  difficult  to  include  radiometric  warping  complications 
in  an  analogue  optical  image  registration  system.  Nonetheless,  the  potential  speed,  power 
and  size  advantages  of  JTCs  over  digital  computers,  makes  it  worthwhile  to  conduct  an 
objective  assessment  of  the  potential  of  JTCs  to  implement  precision  image  registration. 

In  general,  the  content  of  this  report  consists  of  an  impartial  appraisal  of  JTC  and 
associated  optics  research  that  has  been  published  in  the  scientific  literature,  from  the 
perspective  of  precision  image  registration.  It  is  not  the  author’s  ambition  to  materially 
advance  particular  components  of  this  research.  Rather,  the  author’s  intention  is  to  assess 
the  merits  of  individual  technologies  for  inclusion  in  analogue  optical  precision  image  reg¬ 
istration  systems,  and  to  consider  the  prospective  registration  performance  of  conceivable 
system  architectures.  In  a  sense,  this  report  communicates  a  gedankenexperiment  in  op¬ 
tical  system  design.  The  author  does  not  elucidate  the  physical  principles  of  operation  of 
specific  optical  technologies.  To  do  so  would  digress  too  far  from  the  central  theme  of  this 
report,  and  such  information  is  available  to  varying  degrees  of  detail  from  the  appropriate 
journal  articles  that  are  cited  within  this  report.  However,  a  descriptive  account  of  the 
operation  of  conventional  JTCs  is  included,  in  recognition  of  the  pivotal  role  of  the  JTC 
in  any  optical  image  registration  system.  There  is  too  much  uncertainty  regarding  the 
performance  of  individual  optical  subsystems  to  enable  a  definitive  quantitative  assess¬ 
ment;  hence  the  assessment  is  qualitative  of  necessity,  but  nevertheless  is  founded  upon 
defensible  reasoning. 

A  concise  summary  of  the  operation  of  JTCs,  and  the  properties  and  performance  of 
the  many  variants  of  JTCs  that  have  been  devised,  will  form  the  content  of  Section  2. 
In  Section  2  one  will  consider  only  JTCs  that  are  “conventional”,  in  the  sense  of  only 
implementing  the  mathematical  correlation  operation.  The  meaning  and  requirements  of 
precision  image  registration  will  be  reaffirmed  and  elaborated  upon  in  Section  3.  This 
discussion  will  convey  an  appreciation  of  the  enhancements  that  need  to  be  made  to 
conventional  JTCs  to  improve  their  suitability  for  precision  image  registration.  A  review 
of  possible  techniques  for  realising  these  JTC  enhancements,  and  an  assessment  of  their 
potential  for  implementation,  forms  the  content  of  Section  4.  In  Section  5,  conclusions  are 
drawn  about  the  potential  of  JTCs  to  be  used  in  precision  image  registration,  and  indeed, 
which  types  of  JTC  are  the  most  promising  for  this  application. 


2  Properties  of  joint  transform  correlators 


This  section  contains  a  summary  review  of  the  properties  and  performance  of  JTCs, 
primarily  condensed  from  the  detailed  review  articles  of  Javidi  [4,  5]. 

JTCs  are  coherent  optical  systems  with  the  generic  architecture  illustrated  in  Fig¬ 
ure  1,  and  described  in  this  paragraph.  The  reference  and  input  images  are  juxtaposed 
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on  an  image  plane  SLM^.  This  is  followed  by  a  Fourier  transform  lens,  with  a  position 
sensitive  intensity  detector  located  in  the  back  focal  plane  of  the  lens.  Upon  illumina¬ 
tion  by  coherent  light,  the  intensity  distribution  sensed  by  the  Fourier  plane  detector  is 
impressed  upon  a  Fourier  plane  SLM,  subject  to  a  point  transformation  of  the  intensity 
(ie.  the  transformation  at  a  point  is  only  a  function  of  the  intensity  at  that  point).  An 
appropriate  optically  addressed  SLM,  such  as  a  liquid  crystal  light  valve,  combines  the 
intensity  detection,  transformation  and  spatial  light  modulation  functions  into  one  device. 
The  Fourier  plane  SLM  is  followed  by  a  Fourier  transform  lens,  which  has  an  electronically 
readable  position  sensitive  intensity  detector  in  its  back  focal  plane.  Upon  illumination  of 
the  second  stage  by  coherent  light,  one  obtains  a  radiation  distribution  in  the  correlation 
plane  which  is  peaked  at  positions  representing  the  translation  between  similar  objects  in 
the  two  input  images.  For  a  linear  intensity  transformation  between  the  Fourier  plane  de¬ 
tector  and  Fourier  plane  SLM  (ie.  linear  JTC),  the  intensity  distribution  in  the  correlation 
plane  is  precisely  the  square  of  the  correlation  function  of  the  images. 

Depending  on  the  form  of  the  Fourier  plane  intensity  transformation,  one  may  distin¬ 
guish  between  classical  linear  JTCs,  general  nonlinear  JTCs  and  binary  nonlinear  JTCs. 
Nonlinear  JTCs  have  the  following  advantages  over  linear  JTCs  for  image  registration: 
light  utilisation  efficiency,  correlation  peak  height,  sensitivity  to  image  differences  and 
peak  to  clutter  ratios.  Binary  JTCs  exhibit  these  advantages  to  the  greatest  extent  of 
any  nonlinear  JTC.  The  sharpness  of  the  correlation  peaks,  and  hence  their  observability, 
increases  from  linear  to  nonlinear  to  binary  JTCs. 

Binary  JTCs  have  the  additional  practical  advantage  of  being  amenable  to  direct  imple¬ 
mentation  by  binary  Fourier  plane  SLMs,  as  well  as  continuous  amplitude  SLMs.  Although 
linear  and  nonlinear  JTCs  also  may  be  implemented  with  binary  Fourier  plane  SLMs  (see 
later  for  elaboration),  there  is  a  complexity  penalty  for  doing  so,  and  they  are  more  suited 
to  the  use  of  continuous  amplitude  SLMs.  Given  that  the  speed  of  operation  of  any  JTC  is 
limited  by  the  slowest  SLM  response  time,  the  presence  of  a  continuous  amplitude  Fourier 
plane  SLM  in  the  JTC  will  restrict  the  possible  frame  rate  to  below  that  attainable  from 
two  binary  SLMs.  This  contention  is  premised  on  the  generalisation  that  in  the  past, 
continuous  amplitude  SLMs  have  had  slower  response  times  than  binary  SLMs.  However, 
the  advent  of  continuous  amplitude  ferroelectic  liquid  crystal  SLMs  is  beginning  to  remove 
this  disparity. 

The  Fourier  plane  intensity  transformations  used  in  nonlinear  JTCs  tend  to  have  the 
“sigmoid”  shape,  being  linear  at  low  intensity,  and  saturating  at  high  intensity.  Since 
the  spatial  frequency  spectrum  of  practical  images  always  is  greatest  at  low  frequencies 
and  declines  at  higher  frequencies,  a  sigmoid  spectrum  transformation  has  the  effect  of 
emphasising  high  spatial  frequencies  relative  to  low.  Accordingly,  the  influence  of  image 
fine  structure  on  the  Fourier  plane  spectrum  is  enhanced.  This  explains  why  the  JTC 
sensitivity  to  image  differences  increases  with  increasing  severity  of  nonlinearity  of  the 
Fourier  plane  intensity  transformation. 

All  of  the  JTC  characteristics  described  so  far  indicate  that  precision  image  registration 
is  best  accomplished  by  binary  JTCs,  followed  by  other  nonlinear  JTCs  in  preference  to 

^SLM  =  Spatial  Light  Modulator 
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Figure  1:  Basic  architecture  of  the  nonlinear  joint  transform  correlator  (JTC).  Electrically 
addressed  spatial  light  modulators  (SLMs)  are  used,  f  is  the  focal  length  of  the  Fourier 
transform  lenses.  Detectors  are  position  sensitive. 
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linear  JTCs.  However,  even  within  the  binary  JTC  constraint,  there  is  still  much  scope 
for  variation  in  the  JTC  specification  and  design.  SLM  and  detector  device  types,  system 
architecture  and  threshold  specification  are  among  the  remaining  undetermined  factors. 

Javidi  [4]  contends  that  the  binary  JTC  gives  the  best  immunity  to  noise  of  all  three 
JTC  types,  when  the  noise  bandwidth  is  less  than  the  image  bandwidth.  However,  in 
focal  plane  array  sensed  images,  the  detector  noise  component  of  the  total  image  noise  is 
spatially  uncorrelated,  so  the  noise  bandwidth  will  be  greater  than  the  image  bandwidth. 
Consequently,  the  correlation  peak  to  clutter  ratio  is  reduced  by  image  noise  to  a  greater 
extent  in  the  binary  JTC  than  other  JTC  types.  The  consequences  of  this  behaviour  are 
ameliorated  by  the  fact  that  the  binary  JTC  has  the  highest  noise-free  peak  to  clutter 
ratio.  As  long  as  the  image  noise  is  sufficiently  small,  the  binary  JTC  should  retain 
its  peak  to  clutter  ratio  advantage  over  the  other  JTC  types.  However,  the  relatively 
severe  degradation  of  performance  of  the  binary  JTC  with  noise,  makes  it  an  unattractive 
proposition  for  noisy  image  registration.  It  is  not  clear  that  other  JTC  types  are  any 
better. 

The  choice  of  binary  JTC  leaves  unresolved  the  specification  of  the  Fourier  plane 
threshold  criterion.  Three  thresholding  criteria  have  been  addressed  by  Javidi  [4].  Optimal 
thresholding,  which  maximises  the  first  order  correlation  peak,  determines  the  threshold 
as  a  function  of  spatial  frequency;  in  particular,  it  is  the  sum  of  the  spectral  densities  of 
the  input  and  reference  images.  Every  time  an  image  is  changed  a  new  threshold  function 
must  be  determined  by  evaluating  a  Fourier  transform.  At  the  expense  of  increased  JTC 
system  complexity,  this  could  be  accomplished  optically.  Alternatively,  at  the  expense  of 
increased  total  correlation  time,  it  could  be  accomplished  by  digital  computer.  Median 
thresholding  is  also  computationally  intensive;  without  the  option  of  exploiting  the  existing 
JTC  optical  system  to  apply  this  criterion,  instead  of  digital  computation.  Subset  median 
thresholding  is  a  reduced  computation  variation  of  the  median  thresholding  criterion. 

Actually,  immense  leverage  may  be  gained  by  judiciously  choosing  the  binary  JTC 
Fourier  plane  threshold  function.  By  doing  so,  the  binary  JTC  can  emulate  an  arbitrary 
linear  or  nonlinear  JTC.  Thus,  one  has  a  binary  encoded  JTC.  This  is  convenient,  since 
one  may  wish  to  utilise  a  fast  binary  SLM  in  the  Fourier  plane,  instead  of  a  slower  con¬ 
tinuous  intensity  SLM.  The  thresholding  to  achieve  this  is  a  function  of  spatial  frequency, 
depending  on  the  spectral  densities  of  the  reference  and  input  images,  and  on  the  required 
nonlinear  transformation.  This  introduces  an  additional  digital  computation  requirement 
into  the  JTC  system,  which  is  quite  substantial  and  ongoing,  since  the  threshold  function 
is  recomputed  every  time  an  image  changes.  The  extra  computation  time  possibly  cancels 
the  speed  benefits  accruing  from  the  use  of  a  fast  binary  Fourier  plane  SLM,  hence  the 
benefits  of  this  scheme  are  questionable. 

Binary  encoded  JTCs  encounter  particularly  severe  problems  in  emulating  JTCs  that 
are  nearly  linear.  In  this  case,  the  higher  order  correlation  terms  in  the  binary  encoded 
JTC  are  of  similar  magnitude  to  the  desired  first  order  correlation  terms.  It  is  imperative 
that  these  higher  order  correlations  be  well  separated  from  the  first  order,  thus  challenging 
the  SBWPs^  of  both  image  plane  and  Fourier  plane  SLMs,  and  detector  in  the  latter 

®SBWP  =  Space-Bandwidth  Product  (along  one  axis  of  a  pixelated  image,  is  just  one  half  the  number 
of  pixels  along  that  axis) 
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case.  Although  reducing  the  image  resolution  relaxes  the  SBWP  requirement  for  the 
image  plane  SLM,  no  such  reprieve  is  available  for  the  Fourier  plane  detector  and  SLM. 
A  conventional  nonlinear  JTC  with  a  continuous  intensity  Fourier  plane  SLM  has  the 
advantage  of  attaining  the  same  correlation  fidelity  with  the  use  of  lower  SBWP  SLMs. 

In  the  presence  of  several  identical  regions  in  the  reference  image,  linear  and  nonlinear 
JTCs  are  susceptible  to  confusing  and  possibly  ambiguous  correlation  plane  distributions. 
This  is  because  the  correlation  plane  contains  peaks  corresponding  to  mutual  correlation 
between  separate  identical  regions  in  the  reference  image,  as  well  as  the  desired  peaks 
resulting  from  cross  correlation  between  the  reference  and  input  images.  The  presence  of 
these  diversionary  correlation  peaks  further  detracts  from  the  desirability  of  using  linear 
or  nonlinear  JTCs  for  image  registration. 

Binary  JTCs  in  general  are  afflicted  by  the  same  multiple  identical  region  problems. 
However,  optimally  thresholded  binary  JTCs  are  devoid  of  correlation  peaks  between  iden¬ 
tical  regions  of  the  reference  image.  The  correlation  plane  of  optimally  thresholded  binary 
JTCs  contains  only  significant  peaks,  that  is,  those  resulting  from  the  cross  correlation 
between  reference  and  input  images.  This  characteristic  of  optimally  thresholded  binary 
JTCs  makes  interpretation  of  their  correlation  plane  distribution  much  easier  than  for  any 
other  type  of  JTC.  Another  feature  of  optimally  thresholded  binary  JTCs  that  further 
accentuates  the  “cleanliness”  of  the  correlation  plane,  is  that  these  JTCs  are  also  devoid  of 
even  order  correlation  peaks.  This  has  the  effect  of  reducing  the  clutter  that  interferes  with 
the  desired  first  order  cross  correlation  peaks.  These  two  unique  properties  of  optimally 
thresholded  binary  JTCs,  suggest  that  the  optimally  thresholded  binary  JTC  should  be 
the  favoured  type  for  image  registration. 

Javidi  [4]  reports  the  results  of  experiments  conducted  on  a  JTC  with  a  variable  Fourier 
plane  nonlinear  intensity  transformation.  These  experiments  confirm  that  as  the  severity 
of  the  nonlinearity  increases,  in  the  correlation  plane  one  observes  an  increase  in  cross 
correlation  peak  intensity,  an  increase  in  peak  to  clutter  ratio,  an  increase  in  peak  to  side- 
lobe  ratio,  and  a  decrease  in  correlation  peak  width.  Further  experiments  on  the  same 
JTC  demonstrate  that  the  JTC  becomes  more  sensitive  to  scale  and  orientation  changes 
between  otherwise  identical  image  regions,  with  increasing  severity  of  Fourier  plane  non¬ 
linearity.  This  is  detrimental  for  low  fidelity  image  registration  (eg.  target  recognition), 
but  beneficial  for  high  fidelity  image  registration.  The  experimental  results  demonstrate 
that  the  correct  scale  and  orientation  produces  a  maximum  in  the  peak  to  sidelobe  ratio  in 
the  correlation  plane,  relative  to  other  scales  and  orientations.  A  problem  with  observing 
the  peak  to  sidelobe  ratio  is  that  it  is  a  nonlocal  quantity,  whose  neighbourhood  is  not 
well  delineated,  and  substantial  computational  effort  needs  to  be  expended  in  its  evalua¬ 
tion.  One  is  again  confronted  by  the  necessity  to  imbue  the  analogue  optical  JTC  with 
formidable  digital  computational  power;  and  even  then,  it  may  still  be  the  digital  com¬ 
putation  requirements  that  limit  the  speed  of  operation  of  the  whole  image  registration 
system,  especially  if  both  SLMs  have  fast  response  times. 

The  image  plane  SLM  in  a  JTC  must  have  a  high  enough  SBWP  to  accommodate  one 
complete  image  (reference  image),  and  an  image  segment  (input  image),  the  latter  possibly 
also  being  a  complete  image.  There  is  also  a  minimum  allowable  separation  between  the 
reference  and  input  images,  to  ensure  ample  separation  between  autocorrelation  and  cross 
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correlation  peaks.  These  are  particularly  demanding  requirements  for  high  resolution 
imagery.  It  seems  most  likely  that  image  resolution  would  need  to  be  compromised  to 
relax  the  SBWP  requirements  to  within  the  capability  of  present  SLM  technology.  Of 
course,  this  would  reduce  the  precision  of  the  image  registration  performed  by  the  SLM, 
and  there  is  no  scope  for  compensation  by  increasing  the  correlation  time. 

Assuming  that  the  input  image  replicates  some  region  in  the  reference  image,  the  inter¬ 
image  interference  contribution  to  the  Fourier  plane  distribution  takes  the  form  of  high 
spatial  frequency  fringes  modulated  by  a  slowly  varying  envelope.  The  spatial  frequency 
of  the  fringes  is  proportional  to  the  image  plane  separation  between  the  two  identical 
image  segments.  This  spatial  frequency  must  be  within  the  Fourier  plane  detector/SLM 
modulation  transfer  function  pass  band,  otherwise  the  desired  cross  correlation  peak  will 
be  suppressed  relative  to  the  undesired  autocorrelation  peak.  Therefore,  there  is  a  re¬ 
quirement  for  both  the  Fourier  plane  detector  and  SLM  to  have  high  modulation  transfer 
function  bandwidths,  to  ensure  full  resolution  of  the  interference  fringes.  This  requirement 
is  not  diminished  by  reducing  image  resolution.  It  can  only  be  relaxed  by  reducing  the 
size  of  the  reference  image,  or  its  separation  from  the  input  image,  especially  along  the 
direction  of  the  input  image. 

Nonlinear  JTCs  have  pronounced  advantages  over  linear  JTCs  in  the  area  of  SLM 
SBWP  requirements.  The  significantly  narrower  correlation  peak  width  of  nonlinear  JTCs 
relative  to  linear  JTCs  enables  the  reference  and  input  images  to  be  located  closer  to 
one  another  in  the  nonlinear  JTC,  while  still  maintaining  distinguishability  of  the  cross 
correlation  peaks.  This  reduces  the  SBWP  requirements  of  both  the  image  plane  SLM 
and  the  Fourier  plane  SLM. 

If  one  requires  complete  freedom  in  the  specification  of  the  Fourier  plane  nonlinearity 
and  thresholding  criterion,  then  one  must  configure  the  Fourier  plane  as  a  distinct  position 
sensitive  intensity  detector,  followed  by  analogue  or  digital  electronic  processing  elements 
to  effect  the  nonlinear  transformation  including  thresholding,  followed  by  an  electrically 
addressed  SLM.  From  a  systems  perspective,  the  Fourier  plane  is  composed  of  three 
elements,  which  at  present  must  be  implemented  with  three  distinct  devices.  A  single 
optically  addressed  SLM  as  the  Fourier  plane  active  element  does  not  have  the  flexibility 
to  implement  an  arbitrary  nonlinear  transformation.  This  is  unfortunate,  since  from  the 
perspective  of  system  simplicity,  an  optically  addressed  SLM  would  be  the  favoured  option 
for  the  Fourier  plane. 

A  variation  on  the  JTC  architecture  is  to  display  the  reference  and  input  images  on 
separate  SLMs  that  are  axially  separated  (thus  there  are  two  image  planes),  obtaining  the 
class  of  JTCs  known  as  chirp  encoded  JTCs.  For  reasons  to  be  outlined  below,  the  trans¬ 
mission  properties  of  the  two  image  SLMs  must  be  well  matched  to  give  good  correlation 
performance.  The  requirement  for  an  extra  SLM  in  chirp  encoded  JTCs  is  compensated 
by  the  need  for  each  image  JTC  to  have  only  half  the  SBWP  of  an  equivalent  fidelity  con¬ 
ventional  JTC.  Actually,  the  author  has  conceived  a  chirp  encoded  SLM  design  with  the 
input  and  reference  images  in  the  same  plane  (same  SLM  even),  but  with  a  transparent 
phase  retarding  plate  covering  one  of  the  images  on  the  exit  side.  Varying  the  thickness 
of  the  transparent  plate  varies  the  optical  path  difference  between  the  respective  images 
and  Fourier  plane,  which  is  the  same  effect  as  varying  the  physical  axial  separation  of  the 
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two  images  in  the  absence  of  the  plate. 

Chirp  encoded  linear  JTCs  have  three  axially  separated  correlation  planes:  one  for 
the  autocorrelations,  and  two  for  the  two  duplicate  cross  correlations.  Nonlinear  and 
binary  chirp  encoded  JTCs  have  their  higher  order  correlation  terms  focussed  on  different 
planes  yet  again.  In  chirp  encoded  JTCs,  the  correlation  plane  detector  is  placed  in 
coincidence  with  a  plane  in  which  one  of  the  cross  correlation  terms  is  in  focus.  All  of  the 
other  correlation  terms  (auto  and  higher  order)  are  out  of  focus  in  this  plane,  and  their 
“peaks”  are  distributed  over  an  extended  area,  with  a  concomitant  intensity  reduction. 
Consequently,  the  other  correlation  terms  do  not  significantly  clutter  the  desired  cross 
correlation  signal. 

The  transverse  separation  between  the  reference  and  input  images  no  longer  has  an 
influence  on  the  performance  of  the  JTC.  This  separation  may  be  minimised  in  an  effort  to 
minimise  the  spatial  frequency  of  the  Fourier  plane  interference  fringes,  and  hence  minimise 
the  SBWP  requirements  for  the  Fourier  plane  SLM.  The  use  of  beam  splitters  allows  the 
tranverse  separation  between  reference  and  input  images  to  be  reduced  to  the  extent  that 
the  two  images  are  optically  overlapping.  Another  benefit  of  chirp  encoded  JTCs  is  that 
correlations  between  identical  regions  in  the  reference  image  manifest  themselves  as  peaks 
in  the  autocorrelation  plane,  and  not  the  cross  correlation  plane. 

The  requirement  for  the  two  image  SLMs  in  a  chirp  encoded  JTC  to  have  well  matched 
light  transmission  properties  is  an  extrapolation  from  the  theoretical  and  experimental 
research  of  Gregory  ei  al  [6]  on  conventional  (ie.  not  chirp  or  binary  encoded)  linear  JTCs. 
Gregory  e<  al  demonstrate  that  a  linear  JTC  with  unequal  light  transmission  through  the 
reference  and  input  image  will  have  only  partial  interference  in  the  Fourier  plane,  so  that 
the  fringe  contrast  is  lower  than  that  for  equal  transmission.  This  is  reflected  in  the 
correlation  plane  by  a  reduction  in  the  intensity  of  only  the  cross  correlation  peaks,  but 
not  the  unwanted  autocorrelation  peaks. 

Certainly  when  the  autocorrelation  and  cross  correlation  peaks  are  focussed  on  the 
same  plane,  as  in  non-chirp  encoded  JTCs,  this  makes  the  identification  of  cross  correla¬ 
tions  more  difficult,  because  of  increased  clutter.  This  introduces  a  requirement  for  the 
two  images  in  the  image  plane  of  the  JTC  to  be  of  similar  “brightness” ,  which  is  not  always 
the  case  in  image  registration.  Preliminary  digital  processing  may  need  to  be  performed 
to  enable  the  JTC  to  achieve  its  full  performance  potential.  This  may  have  an  adverse 
effect  on  the  speed  of  correlation,  and  it  certainly  increases  the  system  complexity.  Chirp 
encoded  JTCs  would  be  less  sensitive  to  unbalanced  image  brightness,  since  in  the  cross 
correlation  plane,  the  autocorrelation  peaks  are  very  diffuse,  and  so  the  level  of  clutter  is 
lower.  However,  even  chirp  encoded  JTCs  would  suffer  a  reduction  in  the  signal  to  clutter 
ratio  in  the  cross  correlation  plane,  which  can’t  be  beneficial. 


3  Elaboration  on  precision  image  registration 

In  this  section  one  expands  and  makes  more  precise  the  discussion  begun  in  Section  1, 
on  the  characteristics  and  requirements  of  precision  image  registration. 
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The  term  “precision  image  registration”  has  connotations  of  the  ability  to  determine 
the  values  of  many  of  the  parameters  that  quantify  the  transformation  of  one  image  to  an¬ 
other.  At  the  most  elementary  level  is  the  component  of  the  transformation  that  represents 
pure  translation.  JTCs  quantify  this  parameter  by  the  location  of  the  cross  correlation 
peak.  Only  one  degree  of  freedom  in  the  JTC  output,  that  is,  the  correlation  peak  shape, 
remains  to  quantify  all  of  the  other  image  transformation  parameters.  The  obvious  way 
to  proceed  in  the  determination  of  some  of  the  remaining  transformation  parameters  is 
to  “navigate”  a  low  dimensional  manifold  of  parameter  space,  while  seeking  the  sharpest 
cross  correlation  peak.  This  entails  distorting  the  input  image  according  to  the  particular 
location  in  parameter  space. 

The  design  philosophy  of  JTCs  is  to  maximise  the  amount  of  signal  manipulation  per¬ 
formed  by  analogue  optics,  and  minimise  the  amount  of  digital  computation.  To  conform 
to  this  design  philosophy,  JTCs  require  controllable  analogue  optical  means  to  transform 
the  input  image.  This  is  an  extremely  demanding  requirement  for  any  transformation 
that  is  more  complicated  than  magnification  and  rotation  of  the  image.  Accordingly, 
one  will  restrict  the  JTC  to  navigating  the  two  dimensional  transformation  parameter 
subspace  represented  by  variable  image  magnification  and  rotation,  with  all  other  image 
transformations  (eg.  shear,  second  and  higher  order  geometric  terms,  and  all  radiometric 
components)  tacitly  assumed  to  be  zero. 

Restricting  consideration  of  image  transformation  only  to  translation,  rotation  and 
magnification  in  principle  does  limit  the  attainable  precision  of  image  registration.  This 
is  certainly  the  case  if  other  transformation  components  are  not  negligible;  for  example,  if 
one  image  is  acquired  with  shear  distortion  due  to  the  scanned  video  line  synchronisation 
steadily  drifting  from  top  to  bottom,  or  the  shear  distortion  that  would  be  present  in 
the  scanned  video  image  of  an  object  moving  rapidly  relative  to  the  frame  rate.  How¬ 
ever,  in  practice  one  expects  that  the  three  transformation  components  represented  by 
translation,  rotation  and  magnification  would  frequently  account  for  essentially  all  of  the 
transformation  between  the  two  images  being  registered.  In  fact,  research  by  Caprari  [3] 
has  demonstrated  that  even  when  the  transformation  between  two  images  has  a  compli¬ 
cated  dependence  on  position,  the  transformation  of  small  regions  (eg.  size  being  1/4  of  the 
full  image  dimensions)  is  very  well  described  by  only  translation  and  rotation;  although  for 
large  fields  of  view  (>100®),  components  of  radiometric  warping  also  become  significant. 
In  imaging  situations  analogous  to  the  one  considered  by  Caprari,  image  magnification 
certainly  is  of  lesser  importance  than  rotation  or  translation,  but  magnification  capability 
will  still  be  sought,  since  it  doesn’t  introduce  conceptual  difficulties  beyond  those  already 
associated  with  image  rotation. 

Thus,  there  are  credible  reasons  to  believe  that  it  is  a  worthwhile  endeavour  to  seek 
a  precision  image  registration  system  that  accommodates  only  translation,  rotation  and 
magnification.  It  is  true  that  a  digital  computer  image  registration  algorithm  could  be 
formulated  to  navigate  a  much  larger  transformation  parameter  subspace  than  any  JTC, 
and  therefore  yield  higher  fidelity  image  registration  than  the  less  flexible  JTC.  However, 
such  fine  precision  would  only  be  available  at  the  expense  of  immense  computational  effort, 
with  a  correspondingly  long  registration  time. 
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4  Incorporation  of  input  image  rotation  and 
magnification  in  JTCs 

Extensive  literature  searching  by  the  author  has  failed  to  find  any  concrete  examples  of 
JTCs  that  incorporate  either  image  rotation  or  magnification,  let  alone  more  complicated 
transformations.  However,  incorporation  of  image  rotation  and  magnification  in  JTCs  is 
not  conceptually  difficult.  In  this  section  one  shall  consider  various  image  rotation  and 
magnification  methods  that  have  been  reported  in  the  literature.  For  image  registration, 
it  is  the  input  image  segment  that  is  transformed,  while  the  reference  image  remains 
invariant. 

The  following  cautionary  remark  needs  to  be  emphasised  for  image  magnification. 
Unlike  rotation,  geometrical  magnification  also  scales  the  image  intensity,  by  the  amount 
necessary  to  conserve  total  image  energy.  One  may  encounter  the  situation  of  correlating 
two  images  of  different  brightness,  which,  based  on  the  research  of  Gregory  ei  al  [6],  was 
identified  in  Section  2  as  being  problematical.  Thus,  there  is  a  limit  to  the  amount  of 
image  magnification  that  can  be  tolerated  by  a  JTC  before  compensatory  action  must 
be  taken  on  the  intensity.  Nevertheless,  there  is  still  a  large  class  of  image  registration 
applications  in  which  the  mutual  magnification  is  quite  small. 

Jutamulia  and  Asakura  [7]  propose  a  JTC  in  which  a  transparency  displaying  the 
input  image  is  mechanically  rotated,  while  an  SLM  displaying  the  reference  image  is  kept 
stationary.  Their  design  is  just  conceptual,  with  no  experimental  implementation  reported. 
They  are  interested  in  rotation  invariant  pattern  recognition,  therefore  they  superimpose 
the  correlation  outputs  for  all  orientations  to  obtain  the  single  correlation  plane  light 
distribution  of  interest.  In  contrast,  for  image  registration  one  must  distinguish  between 
correlation  plane  distributions  corresponding  to  different  input  image  orientations.  In  the 
proposed  scheme,  the  input  image  is  displayed  on  a  photographic  transparency,  so  there 
are  no  cables  to  impede  rotation.  The  disadvantage  of  this  arrangement  is  that  producing 
and  inserting  new  input  images  is  a  procession  of  manual  operations,  not  at  all  suited  to 
real  time  implementation.  Any  registration  system  with  reaisonably  fast  throughput  must 
have  the  input  image  displayed  on  a  rotating  SLM. 

The  inertia  of  the  SLM  about  its  rotation  axis  would  be  quite  high,  especially  con¬ 
sidering  that  there  would  need  to  be  a  fairly  long  unsupported  span  of  power  and  signal 
cables  hanging  off  of  the  SLM  and  rotating  with  it.  This  restricts  the  angular  accelerations 
that  would  be  achieved  during  SLM  reorientations,  so  that  any  reorientations  take  longer 
to  execute.  Alignment  and  positioning  of  the  SLM  rotation  axis  would  be  of  the  utmost 
importance.  The  continual  flexing  of  cables  connected  to  the  SLM  as  it  rotates  could  lead 
to  fatigue  failure  of  the  cables.  There  would  be  a  maximum  angle  through  which  the  SLM 
could  be  rotated,  beyond  which  the  contortion  of  the  cables  becomes  perilously  great.  To 
penetrate  this  barrier,  the  SLM  would  actually  have  to  be  rotated  through  almost  a  whole 
revolution  in  the  opposite  sense.  This  is  an  inefficient  procedure,  since  a  very  large  rota¬ 
tion  is  undrtaken  to  achieve  a  very  small  effective  orientation  change.  Thus,  the  design 
of  Jutamulia  and  Asakura  is  the  most  obvious  and  simplistic  method  of  incorporating 
image  rotation  in  the  JTC,  but  it  may  be  susceptible  to  cable  deterioration  and  alignment 
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degeneration  with  use.  The  design  is  also  inherently  slow  in  executing  image  rotation,  be¬ 
cause  this  is  accomplished  by  mechanical  means.  Its  main  redeeming  feature,  apart  from 
conceptual  simplicity,  is  that  there  is  no  deterioration  of  image  quality  with  rotation. 

Image  rotation  and  magnification,  and  indeed  a  much  wider  range  of  geometric  trans¬ 
formations,  can  be  effected  by  the  interferometric  technique  invented  by  Bryngdahl  [8,  9]. 
This  entails  multiplying  the  amplitude  image  in  the  front  focal  plane  of  a  lens  by  a  judi¬ 
ciously  chosen  phase  distribution,  so  that  the  required  transformed  image  appears  in  the 
back  focal  plane  of  the  lens.  Bryngdahl’s  scheme  essentially  is  to  form  the  Fraunhofer 
diffraction  pattern  of  an  amplitude  object  that  has  been  phase  modified.  It  is  this  phase 
modification  that  causes  the  diffraction  to  emulate  a  point  transformation. 

In  Bryngdahl’s  pioneering  research,  and  that  of  many  following  researchers,  the  phase 
variation  superimposed  on  the  input  image  is  obtained  by  overlaying  either  a  kinoform 
or  computer  generated  hologram  onto  the  image.  Such  phase  filters  are  not  amenable  to 
electronic  or  optical  control,  requiring  the  physical  removal  and  insertion  of  phase  plates 
for  variation.  They  are  thus  inflexible.  The  image  transformations  experimentally  achieved 
by  Bryngdahl  are  very  aberrated,  and  entirely  unsuitable  for  precision  image  registration 
purposes. 

The  interferometric  image  transformation  technique  of  Bryngdahl  is  quite  elegant,  but 
it  only  works  under  some  fairly  liberal  constraints.  One-to-one  mappings  result  only  if 
the  input  image  has  a  sufficiently  narrow  spatial  frequency  bandwidth.  For  images  with 
high  spatial  frequency  structure,  the  transformed  image  is  expected  to  be  aberrated  even 
more  than  the  already  low  quality  examples  given  by  Bryngdahl.  In  particular,  sharp 
edges  in  binary  images  would  be  smeared  out  to  some  extent  in  the  transformed  image; 
possibly  even  fringing  would  occur  in  the  vicinity  of  the  edges.  Furthermore,  rapid  spatial 
variation  of  the  input  image  amplitude  renders  the  geometric  transformation  dependent 
on  the  structure  of  the  input  image,  which  is  certainly  not  desired.  Any  phase  variation 
across  the  input  image  area,  in  addition  to  the  deliberately  imposed  phase  filter,  will 
also  reduce  the  quality  of  the  transformed  image  even  further.  It  is  important  to  be 
cognisant  of  this  effect,  because  some  supposedly  amplitude  modulating  SLMs  also  have  an 
associated  phase  modulation.  In  summary,  there  is  plenty  of  scope  for  the  interferometric 
image  transformation  technique  to  be  degraded  further  than  it  is  in  the  examples  given 
by  Bryngdahl. 

The  above  interferometric  image  transformation  system  is  based  on  a  Fourier  trans¬ 
forming  optical  system.  An  alternative  interferometric  image  transformation  system  sug¬ 
gested  by  Bryngdahl  is  based  upon  an  imaging  optical  system  composed  of  one  or  more 
lenses.  Such  a  system  needs  an  extra  diffractive  or  refractive  element  in  addition  to  the 
phase  filter  that  realises  the  image  transformation.  In  effect  it  causes  two  transformations. 
Bryngdahl  makes  no  claims  of  superiority  for  this  more  complicated  system. 

The  Bryngdahl  interferometric  scheme  is  complicated  by  the  limited  types  of  image 
transformations  that  can  be  implemented  by  a  single  phase  filter.  For  example,  neither 
pure  image  rotation  nor  pure  image  shear  are  attainable  from  a  single  phase  filter.  Two 
interferometric  transformation  optical  systems  must  be  cascaded  to  obtain  pure  image 
rotation,  with  a  corresponding  doubling  of  system  size,  complexity  and  cost.  The  first 
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stage  uses  a  phase  filter  that  performs  a  combined  rotation  about  the  normal  to  the 
image  plane,  and  reflection  in  an  axis  within  the  image  plane.  To  cancel  the  reflection, 
the  second  stage  must  have  a  phase  filter  that  realises  only  the  same  reflection  as  in 
the  first  stage.  The  need  for  a  two  stage  interferometric  filter  patently  detracts  from 
the  attractiveness  of  interferometric  image  rotation,  especially  considering  the  low  quality 
experimental  results  obtained  by  Bryngdahl  for  a  single  stage.  Image  magnification  may 
be  incorporated  into  the  first  stage  interferometric  transformation  without  the  need  for 
further  optical  componentry;  it  only  requires  appropriate  modification  to  the  phase  filter. 

To  obtain  variable  rotation  and  magnification  abilities  in  the  interferometric  system, 
the  phase  filter  must  be  made  variable.  A  physical  mechanism  with  a  proven  capacity  to 
achieve  this  property  is  the  electrooptic  effect,  in  which  the  refractive  index  distribution 
across  a  film  of  suitable  material  responds  to  the  nonuniform  macroscopic  electric  field 
within  the  material.  A  very  suitable  material  for  exploiting  the  electrooptic  effect  is  a 
liquid  crystal  film.  The  electric  field  nonuniformity  is  achieved  by  varying  the  voltage 
across  pixelated,  transparent  electrodes  that  have  been  deposited  on  the  two  faces  of  the 
film.  One  has  just  described  liquid  crystal  phase  SLMs,  which  are  a  well  established,  but 
still  progressing,  electrooptical  technology.  It  follows  that  liquid  crystal  phase  SLMs  have 
prospects  for  use  as  electronically  configurable  phase  filters. 

The  SBWP  limitations  of  the  SLM  phase  filter  will  become  apparent  if  the  required 
phase  filter  fringe  pattern  is  too  fine,  which  it  probably  would  be  if  one  were  to  insist  on 
high  fidelity  image  transformation.  For  each  different  image  transformation,  the  necessary 
phase  filter  fringe  pattern  must  be  recomputed  and  rewritten  by  resetting  the  electrode 
voltages  on  all  pixels;  a  task  that  requires  significant  computational  resources  and  elapsed 
time.  Perhaps  the  most  seductive  property  of  interferometric  image  transformation  by  the 
use  of  an  SLM  phase  filter,  is  that  it  furnishes  true  electronic  image  transformation,  with 
absolutely  no  moving  parts.  Thus,  potentially  it  is  a  very  rugged,  reliable,  and  energy 
efficient  scheme.  Also,  the  absence  of  macroscopic  mechanical  relaxation  times  following 
any  adjustment,  offers  the  opportunity  for  relatively  fast  operation;  limited  only  by  liquid 
crystal  SLM  relaxation  and  digital  computation  times.  The  only  other  variable  image 
transformation  technique  whose  implementation  is  completely  rigid  mechanically,  is  the 
In  r-&  to  cartesian  image  transformation  discussed  below,  which  has  its  own  severe  SBWP 
capacity  problems. 

Braunecker  et  al  [10]  consider  the  use  of  crossed  one  dimensional  Fresnel  zone  plates  as 
diffractive  elements  superimposed  on  the  image  in  the  front  focal  plane  of  a  lens.  The  back 
focal  plane  of  the  lens  contains  the  rotated  and  magnified  image,  the  amount  of  rotation 
and  magnification  depending  on  the  orientations  of  the  two  Fresnel  zone  plates.  Two  off- 
axis  portions  of  Fresnel  zone  plates  are  utilised  in  this  diffractive  scheme,  in  contrast  to  the 
use  of  a  single  hologram  in  the  original  Bryngdahl  [9]  interferometric  scheme.  In  the  two 
Fresnel  zone  plates  scheme,  image  rotation  and  magnification  are  achieved  by  rotating  the 
invariant  zone  plates,  whereas  the  original  holographic  scheme  achieved  the  same  effect 
by  varying  the  fringe  pattern.  The  Fresnel  zone  plates  scheme  performs  rotation  and 
magnification  in  one  optical  stage,  as  opposed  to  the  two  optical  stages  required  for  the 
single  hologram  scheme.  In  common  with  the  single  hologram  technique,  the  two  Fresnel 
zone  plates  technique  restricts  the  spatial  frequency  bandwidth  of  the  image  for  good 


12 


DSTO-TR-0243 


results  to  be  obtainable.  Experimental  results  presented  by  Braunecker  ei  al  indicate 
a  severe  degradation  of  image  quality  resulting  from  this  diffractive  method  of  image 
transformation;  quite  reminiscent  of  the  earlier  interferometric  method  of  Bryngdahl. 

Braunecker  ei  al  [10]  also  present  a  refractive  analogue  of  their  diffractive  system, 
in  which  the  Fresnel  zone  plates  are  replaced  by  adjacent  convex  and  concave  cylindrical 
lenses  of  equal  focal  length  (same  magnitude,  but  opposite  sign).  The  lenses  are  centred  on 
the  optical  axis  in  this  refractive  architecture,  in  distinction  to  the  use  of  off-axis  portions 
of  Fresnel  zone  plates  in  the  diffractive  architecture.  Once  again,  independent  rotation  of 
the  two  lenses  about  their  optical  axis  yields  both  rotation  and  magnification.  Subject  to 
the  absence  of  lens  aberrations,  the  refractive  system  has  the  important  advantage  over 
the  diffractive  and  interferometric  systems,  that  it  rotates  and  magnifies  images  without 
degradation,  no  matter  how  wide  the  spatial  frequency  bandwidth  of  the  images.  Cer¬ 
tainly,  the  experimental  results  presented  by  Braunecker  ei  al  for  the  refractive  technique 
are  much  better  than  those  for  its  diffractive  counterpart,  and  indeed,  the  original  inter¬ 
ferometric  technique.  Braunecker  ei  al  concede  that  their  refractive  scheme  definitely  is 
superior  to  their  diffractive  scheme. 

It  transpires  that  the  cylindrical  lens  image  rotation  and  magnification  scheme  adopted 
by  Braunecker  ei  al  is  ubiquitous,  having  been  included  in  the  earlier  review  of  Swift  [11], 
Swift’s  cylindrical  lens  image  rotator  consists  of  three  cylindrical  lenses,  whereas  Brau¬ 
necker  ei  al  [10],  in  both  their  text  and  figures,  imply  that  the  same  effect  may  be  achieved 
with  just  two  cylindrical  lenses.  However,  the  author  does  not  accept  Braunecker  ei  aVs 
contention  that  two  cylindrical  lenses  alone  are  sufficient  to  rotate  the  image.  The  impor¬ 
tant  parameter  for  image  rotation  action,  as  asserted  by  Braunecker  ei  al,  is  the  orientation 
of  the  bisector  of  the  cylindrical  axes  of  the  two  cylindrical  lenses  (the  cylindrical  axes  are 
perpendicular  to  the  conventional  optical  axis  of  the  lens  system,  which  is  in  turn  normal 
to  the  image  plane,  and  along  which  all  optical  elements  are  aligned).  But  if  all  other  op¬ 
tical  elements  in  the  system  are  rotationally  symmetric  about  the  optical  axis,  or  if  there 
are  no  other  optical  elements,  the  orientation  of  the  bisector  can  not  influence  the  image 
rotation  produced  by  the  optical  system.  Reflection  symmetry  arguments  preclude  the 
possibility  of  image  rotation,  although  the  same  symmetry  considerations  do  not  account 
for  the  absence  of  image  distortion  as  well.  Following  the  scheme  of  Swift,  the  inclusion 
of  a  third  cylindrical  lens  breaks  the  previous  reflection  symmetry,  if  its  cylindrical  axis  is 
neither  parallel  nor  perpendicular  to  the  aforementioned  bisector.  There  is  now  an  identi¬ 
fiable  datum  from  which  the  bisector  direction  is  uniquely  specified,  and  it  is  conceivable 
that  rotating  the  bisector  direction  (i.e.  rotating  the  original  pair  of  cylindrical  lenses  to¬ 
gether)  will  also  rotate  the  image.  Since  Braunecker  ei  al  present  experimental  results 
that  quite  clearly  demonstrate  image  rotation,  one  can  only  conclude  that  their  actual 
optical  architecture  included  at  least  one  non-rotationally  symmetric  optical  element  that 
was  omitted  from  their  discussion  and  diagrams. 

Notwithstanding  the  incompleteness  of  the  description  presented  by  Braunecker  ei  al, 
their  refractive  image  rotation  and  magnification  scheme  demands  attention.  It  has  demon¬ 
strably,  although  not  unequivocal,  good  performance,  and  is  not  too  complex  for  incorpo¬ 
ration  in  practical  JTCs.  It  is  worthy  of  consideration  for  any  application  that  requires 
combined  rotation  and  magnification  of  images,  with  good  retention  of  image  quality.  On 
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the  basis  of  available  information,  and  from  the  perspective  of  present  electrooptical  tech¬ 
nology,  this  double  cylindrical  lens  image  rotation  and  magnification  technique  seems  to 
be  the  best  of  all  the  methods  considered  in  this  report. 

It  is  common  knowledge  among  researchers  that  the  In  r-d  ^  to  cartesian  coordinates 
image  transformation,  converts  mutually  rotated  and  magnified  image  pairs  into  pairs  of 
identical  warped  images  that  are  translated  with  respect  to  one  another.  The  transla¬ 
tion  component  along  the  Inr  axis  yields  the  magnification,  and  the  translation  compo¬ 
nent  along  the  B  axis  yields  the  rotation.  Conventional  correlation  techniques,  such  as  a 
JTC,  in  principle  are  then  able  to  determine  the  translation  components.  Casasent  and 
Psaltis  [12]  are  among  the  proponents  of  this  technique  for  rotation  and  scale  invariant 
pattern  recognition.  The  same  technique  should  be  equally  successful  for  registration 
of  mutually  rotated  and  magnified  images  as  it  is  for  pattern  recognition.  The  In  r-B  to 
cartesian  transformation  is  very  appealing,  because  it  dispenses  with  the  need  to  explicitly 
rotate  and  magnify  the  input  image  in  a  quest  for  the  optimum  rotation  and  magnifica¬ 
tion,  hence  it  endows  conventional  correlators  (such  as  JTCs)  with  the  ability  to  determine 
rotations  and  magnifications.  However,  there  are  severe  problems  with  this  technique. 

One  may  demonstrate  that  the  In  r  to  a;  transformation  increases  the  one  dimensional 
SBWP  along  that  axis  alone  by  a  factor  of  In  Nl'j2  and  that  the  ^  to  y  transformation 
increases  the  one  dimensional  SBWP  along  its  axis  by  a  factor  of  \/2ir  (for  an  indication 
of  a  method  of  derivation,  refer  to  Casasent  and  Psaltis  [13]).  Consequently,  the  lnr-0  to 
cartesian  transformation  increases  the  two  dimensional  SBWP  of  the  image  by  a  factor  of 
TrlnW  17.4  for  a  modest  image  size  of  iV=256,  and  TrlnW  w  8.7  for  a  modest  image 
segment  size  of  W=16.  JTCs  are  inherently  demanding  on  SLM  SBWP,  since  both  the 
reference  image  and  input  image  usually  are  displayed  on  the  same  SLM;  and  in  the  case 
of  precision  image  registration,  the  large,  high  resolution  reference  image  is  particularly 
demanding  of  SBWP  capacity.  Hence,  there  seems  to  be  very  little  scope  for  current 
or  prospective  SLM  technology  to  tolerate  demands  for  up  to  eighteenfold  increases  in 
SBWP  capacity.  To  implement  the  lnr-0  to  cartesian  transformation  on  present  SLMs 
would  necessitate  a  severe  diminution  of  image  resolution,  which  is  contrary  to  the  notion 
of  “precision”  image  registration. 

A  more  fundamental  problem  with  the  In  r-B  to  cartesian  transformation  is  that  it  has 
the  simple  interpretation  articulated  above,  only  when  the  two  images  have  corresponding 
image  points  at  the  origin  of  their  respective  coordinate  systems;  essentially,  there  is  no 
translational  misregistration.  However,  one  does  not  have  prior  knowledge  about  corre¬ 
sponding  pairs  of  points  in  the  two  images  that  are  suitable  coordinate  system  origins.  The 
favoured  scheme  for  overcoming  this  problem,  is  to  register  the  Fourier  transform  mag¬ 
nitudes  of  the  images,  rather  than  the  images  themselves.  Using  the  Fourier  magnitude 
distributions  removes  all  uncertainty  about  appropriate  coordinate  system  centres — one 
uses  the  origins  of  the  Fourier  spaces  of  the  two  images.  Unfortunately,  by  discarding 
the  Fourier  phase  information,  one  forfeits  any  possibility  of  extracting  the  relative  im¬ 
age  translation  vector  concurrently  with  the  magnification  and  rotation.  Casasent  and 
Psaltis  [12]  report  experimental  results  of  a  matched  filter  type  correlator  implementa- 

and  6  are  plane  polar  coordinates 
^for  an  NuN  pixel  untransformed  image 
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tion  of  the  lnr-0  to  cartesian  transformation.  Tentatively,  their  results  seem  to  be  of 
satisfactory  quality,  but  they  are  possibly  assisted  by  the  simplicity  of  the  test  images. 

A  feasible  solution  to  this  problem  could  be  to  determine  the  optimal  magnification  and 
rotation  first,  by  the  Inr-^  to  cartesian  transformation  applied  to  the  Fourier  magnitudes, 
as  described  above.  Then  one  could  magnify  and  rotate  the  input  image  accordingly,  and 
determine  the  translation  by  a  second,  conventional  correlation.  Thus,  two  correlations 
would  need  to  be  performed  for  every  image  registration;  one  to  determine  the  magnifi¬ 
cation  and  rotation,  and  the  other  to  determine  the  two  translation  vector  components. 
This  would  double  the  time  required  to  undertake  the  registration.  Nevertheless,  this 
technique  does  have  appeal,  because  it  is  completely  devoid  of  mechanical  motion  in  the 
JTC,  and  also  because  it  is  the  only  technique  discussed  here  that  determines  all  image 
transformation  parameters  without  recourse  to  an  iterative  search.  Its  most  debilitative 
problem  is  its  extremely  high  SBWP  requirement.  Casasent  and  Psaltis  [12]  propose  an 
alternative,  sophisticated,  but  complicated,  method  of  simultaneously  determining  image 
translation,  rotation  and  magnification.  It  is  not  at  all  obvious  that  their  proposal  has 
merit  as  a  contender  for  a  practical  system. 

The  standard  method  of  optically  rotating  an  image  is  by  the  use  of  a  Dove  prism, 
or  one  of  its  analogues,  as  discussed  by  Swift  [11]  and  Ginsberg  [14].  In  the  context 
of  JTCs,  the  Dove  prism  would  be  inserted  longitudinally  along  the  optical  axis  of  an 
imaging  system  that  precedes  the  conventional  JTC  input  image  SLM.  The  input  image 
to  be  registered  now  would  be  displayed  on  a  screen  in  the  front  focal  plane  of  the  imaging 
system.  It  is  not  necessary  for  the  light  emanating  from  the  input  image  to  be  coherent,  so 
that  any  display  screen  would  be  suitable,  not  just  coherent  display  devices  such  as  SLMs. 
The  rotated  image  would  be  projected  on  a  position  sensitive  detector  in  the  back  focal 
plane  of  the  imaging  system.  Finally,  the  detector  light  distribution  is  impressed  upon 
the  JTC  input  image  SLM,  to  be  correlated  with  the  image  on  the  JTC  reference  image 
SLM.  The  position  sensitive  detector  and  JTC  input  image  SLM  may  be  implemented  as 
a  single  optically  addressed  SLM,  instead  of  two  distinct  devices. 

Rotation  of  the  Dove  prism  around  the  optical  axis  by  a  certain  angle,  causes  the 
projected  image  to  rotate  by  twice  that  angle.  An  adjustable  zoom  lens  arrangement  may 
be  incorporated  into  the  same  imaging  system  that  includes  the  Dove  prism.  The  resulting 
JTC  would  have  mechanically  variable  rotation  and  magnification  of  the  input  image. 
Quality  of  the  transformed  image  should  be  dependent  on  primarily  the  optical  quality  of 
the  Dove  prism  and  zoom  lens.  Dove  prisms  introduce  coma  and  astigmatism  aberrations 
(Bushmelev  et  al  [15]),  which  can  be  partially  corrected  by  appropriate  imaging  lens 
modifications.  Although  the  quality  of  the  transformed  image  is  only  slightly  dependent 
on  the  alignment  of  the  Dove  prism  and  zoom  lens,  the  position  of  the  transformed  image 
in  the  input  image  plane  of  the  JTC  depends  sensitively  on  this  alignment.  It  follows  that 
the  position  of  the  correlation  peak,  but  not  its  magnitude,  also  depends  sensitively  on 
this  alignment,  which  is  a  detrimental  feature  of  this  technique. 

Any  practical  implementation  of  such  a  JTC  would  need  careful  attention  devoted 
to  achieving  mechanical  precision  and  ruggedisation.  This  is  a  challenging  undertaking 
given  the  mechanical  complexity  of  this  scheme.  However,  the  Dove  prism  and  zoom  lens 
technique  for  image  rotation  and  magnification  warrants  attention,  because  none  of  its 
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problems  are  insurmountable  by  the  best  technology  presently  available.  Judged  from  a 
practical  perspective,  probably  this  would  be  the  second  most  favoured  image  rotation 
and  magnification  method,  after  the  double  cylindrical  lens  method  of  Braunecker  ei  al. 


5  Conclusion 

To  the  author’s  knowledge,  a  successful  JTC  based  precision  image  registration  sys¬ 
tem  has  never  been  implemented,  or  at  least  reported  in  the  literature.  Upon  reading 
this  report,  which  addresses  only  the  more  obvious  and  fundamental  problems,  this  should 
not  be  surprising.  Basically,  conventional  JTCs  are  incapable  of  implementing  even  the 
most  rudimentary  image  warping  required  for  precision  image  registration.  And  suitably 
enhanced  JTCs  tend  to  be  mechanically  complicated,  while  requiring  substantive  embed¬ 
ded  digital  computational  power,  thus  defeating  the  purpose  of  analogue  optical  image 
registration. 

Specific  problematical  aspects  of  the  optical  architectures  identified  in  Sections  2  and  4 
include:  the  high  rotational  inertia  of  the  refractive  optical  elements  that  are  rotated, 
whether  they  be  cylindrical  lenses  or  a  Dove  prism;  in  the  case  of  a  rotating  Dove  prism, 
the  axis  of  rotation  is  not  one  of  its  dynamical  principal  axes;  the  susceptibility  of  the 
drive  mechanisms  for  the  mechanical  degrees  of  freedom  to  backlash;  the  complicated 
nature  of  the  correlation  plane  light  distribution  for  some  JTC  architectures,  with  its 
attendant  requirement  for  heavy  computation  to  extricate  the  desired  cross  correlation 
peaks;  the  possible  need  to  compute  nonlinear  JTC  Fourier  plane  threshold  functions;  the 
need  to  instigate  a  sophisticated,  two  dimensional  rotation  and  magnification  parameter 
space  optimisation  strategy,  which  involves  much  computation;  the  centrality  of  optical 
to  analogue  electronic  to  digital  conversion  processes,  and  vice-versa,  with  their  atten¬ 
dant  response  times;  and  the  relatively  long  relaxation  times  of  the  mechanical  transient 
responses. 

If  one  were  to  attempt  the  design  and  construction  of  a  JTC  based  precision  image 
registration  system,  then  this  report  would  assist  one  in  forming  a  distinct  preference 
for  particular  system  architectures  over  all  other  alternatives.  Based  upon  the  survey  of 
conventional  JTCs  that  is  undertaken  in  Section  2,  it  seems  that  the  most  promising  JTC 
architecture  is  the  chirp  encoded  binary  variety;  with  a  Fourier  plane  threshold  function 
that  is  preferably  invariant,  or  at  worst,  simple  to  compute  (i.e.  definitely  not  optimal 
thresholding).  Additionally,  one  would  configure  the  reference  and  input  images  to  be 
optically  overlapping,  which  is  a  trivial  matter  if  a  beam  splitter  is  included  (which  it 
probably  would  be,  regardless  of  this  requirement).  As  explained  in  Section  3,  in  deference 
to  the  difficulty  of  optically  warping  images,  one  enhances  the  conventional  JTC  only 
by  including  image  magnification  and  rotation  capabilities.  Based  upon  the  survey  of 
optical  image  rotation  techniques  that  is  undertaken  in  Section  4,  it  seems  that  the  most 
promising  technique  for  optically  rotating  and  magnifying  images  is  the  double  rotating 
cylindrical  lens  scheme  of  Braunecker  et  al  [10].  The  next  best  optical  image  rotation  and 
magnification  technique  seems  to  be  the  “obvious”  one  of  a  Dove  prism  and  zoom  lens 
combination. 
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Any  such  optical  precision  image  registration  system  would  have  two  characteristics 
that  substantively  nullify  the  presumed  advantages  of  JTCs.  The  mechanical  adjustability 
of  the  system  makes  it  bulkier,  more  complicated,  slower,  less  rugged,  less  energy  efficient 
and  less  reliable  than  an  unadorned  JTC.  Compounding  some  of  these  problems,  is  the 
inclusion  of  considerable  electronic  control  and  computer  hardware,  that  would  not  be 
present  in  unadorned  JTCs.  Given  that  the  principal  reason  for  seeking  an  analogue 
optical  approach  to  precision  image  registration  was  to  avoid  the  use  of  a  digital  computer 
for  anything  other  than  simple  operations,  one  would  conclude  that  this  quest  has  failed. 
The  JTC  design  that  shows  the  most  promise,  has  an  embedded  digital  computer  as  an 
indispensable  element;  and  if  that  doesn’t  impede  the  rate  of  registrations  too  much,  then 
the  mechanical  adjustments  certainly  will! 

An  objective  appraisal  of  the  capabilities  of  optical  correlators  in  the  context  of  the 
requirements  of  precision  image  registration,  affords  one  little  enthusiasm  for  using  the 
former  to  accomplish  the  latter.  Digital  computers  are  much  more  adaptable  towards 
the  sophisticated  operations  required  for  precision  image  registration.  Since  the  precision 
image  registration  problem  is  expressed  mathematically  (with  no  scope  for  singularities  or 
diverging  integrals  or  series),  there  is  a  computational  algorithm  that  solves  the  problem 
to  arbitrary  precision,  and  one  may  conceive  of  a  digital  computer  that  can  execute  this 
algorithm.  The  only  remaining  issue  is  the  time  taken  for  the  solution  of  the  problem. 
Even  if  practicable  digital  computers  are  too  slow  for  the  application  that  is  envisaged, 
given  the  above  revelations,  there  is  not  much  reason  to  believe  that  practicable  optical 
precision  image  registration  systems  are  any  faster.  Analogue  optical  systems  certainly 
would  not  be  more  precise  than  digital  computers. 

Based  upon  the  current  state  of  analogue  optical  versus  digital  electronic  technology, 
it  is  the  author’s  considered  opinion  that  it  would  be  more  fruitful  to  appeal  to  the  latter 
technology  in  tackling  the  problem  of  precision  image  registration. 
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